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Susan Elswick, University of Memphis 
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I n a time when parents are presented with many elementary and second¬ 
ary schooling options, public and private schools often find themselves in 
a heated competition to attract and keep students. Parents choosing be¬ 
tween educational options often consider a multitude of criteria before choos¬ 
ing an appropriate setting including the schools religious foundations, cost, 
and proximity to the family residence. More and more, however, new metrics 
have become part of this parental decision. With the desire to improve educa¬ 
tion across the United States, the No Child Left Behind act (NCLB, 2002) 
required a greater transparency in the reporting of academic achievement data 
than was previously required of pubic schools. While private, non-government 
funded, faith-based institutions were not required to commit to the same level 
of transparency, they often felt the need to do so in order to keep up with the 
information demands of parents considering the private schools as educational 
options for their child. 

Teachers in Catholic schools are not immune from pressures to improve 
students’ scores on high stakes tests and standards-based education is not 
new to Catholic schools. Nationally, many public school systems have moved 
to implement Common Core State Standards (CCSS) or other similar stan¬ 
dards. Assessment, in turn, has been tied to these standards. In many states, 
Catholic schools have the option as to whether to implement accepted state 
standards or to create standards of their own. The Committee on Catholic 
Education (CCE) gives each bishop the authority to lead the CCSS discus¬ 
sion at the local level and to eventually make a decision based on what is best 
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for each individual diocese (2014). Regardless of implementation of govern¬ 
ment mandates, students graduating from Catholic schools will ultimately 
be held to the same standards and outcome based measures when it comes 
to their graduates’ ability to compete for post-secondary educational scholar¬ 
ships, national merits, and other accolades based in part on national stan¬ 
dardized test scores. 

The academic achievement data provided by public school systems allow 
parents and others to see if their students meet grade level standards. Catho¬ 
lic schools then, must provide such data if they wish to measure themselves 
against those enrolled in the competing public and private schooling op¬ 
tions. Grades are no longer the sole measure of academic progress and do not 
suffice to provide an on-going measure of student achievement. Academic 
progress during the school year is a required measure of predicted summative 
achievement results. Therefore, data collection and analysis at the individual 
classroom and student levels during daily instruction is necessary to ensure 
competitive, school-wide scores will be achieved following any academic year. 

Data collection and utilization requires that teachers collect information 
within both a formative and summative format from consistent assessment 
of their students. Unfortunately, teachers are often at a loss as to how to ef¬ 
fectively collect and utilize data as a driving force within their educational 
assessment and planning routines (Marshall, 2009; Young & Kim, 2010). This 
is true within Catholic schools, as well as in competing private, public, and 
charter institutions. If Catholic schools are to remain a viable option in this 
ever-increasing competitive educational environment, teachers must become 
more comfortable collecting and analyzing data in an effort to help drive 
instruction and high levels of academic achievement. 

The collection of data to drive instruction is often referred to as data- 
informed instruction (DII). Marsh, Pane, and Hamilton (2006) define DII 
as the practice of teachers and administrators systematically collecting and 
analyzing a variety of data to guide instructional decisions and advance the 
performance of students and schools. The premise behind DII is to imbed 
data collection into daily classroom routines, use the data to make any neces¬ 
sary instructional plans or modifications, and to continuously monitor stu¬ 
dent performance to predict academic gains. When DII is used properly, it 
becomes an ongoing formative assessment. These formative assessments allow 
progress monitoring at the individual, class-wide and building levels, while 
providing an opportunity for teachers to adjust teaching strategies and make 
instructional modifications as needed (Sattler,200i). 
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The focus for a DII model is to use data to shape instructional decisions 
and increase student success. In a successful DII model, teachers must possess 
knowledge and specific skills related to: data analysis, data interpretation, and 
the use data to make instructional decisions. Researchers have shown that 
DII practices are often unfamiliar to teachers and are infrequent topics of 
focus within teacher training and professional development programs (Mar¬ 
shall, 2009; Young & Kim, 2010). Therefore, teachers often feel unprepared 
to use assessment effectively. This is true in public, private, and charter school 
educational settings. According to a report from the U.S. Department of 
Educations Office of Planning, Evaluation and Policy Development (Means, 
Chen, DeBarger, & Padilla, 2011), teachers are challenged and frustrated 
when working with high-stakes test data. This report showed that teachers’ 
struggles persisted from simple to complex tasks involving the use of data 
to adapt instruction. Means et al. (2011) also found that while most teachers 
could locate data on a table with ease, they struggled when asked to identify 
specific data. Likewise, teachers were not able to conduct comparison cal¬ 
culations. In most cases, teachers overlooked critical data points or became 
overwhelmed by the abstract nature of data analysis. Lacking basic analytic 
skills made it difficult for the teachers to understand the data in a table and 
justify the reasoning behind analyses (Means et al., 2011) that may have been 
critical in influencing daily academic planning. 

The challenge of preparing teachers and administrators to use DII is felt 
in public and private educational settings. Teacher training programs typical¬ 
ly introduce data analysis and interpretation skills in isolation. This creates a 
dichotomy that generally does not provide an infrastructure for understand¬ 
ing the integration of formative and summative assessment into daily in¬ 
structional decisions or how to use data-informed decision-making strategies 
to positively impact student learning. Therefore, some burden is placed on 
school districts to provide professional development that focuses on develop¬ 
ing each teacher’s data analytic and interpretation skills as well as focuses on 
how to use data in the instructional planning process. 

Assessment Guides Teaching 

Data-informed instruction is intended to utilize ongoing data collection 
and analysis to closely approximate students’ rates of academic skill acquisi¬ 
tion and knowledge (Cizek, Fitzgerald, & Rachor, 1996; Herman & Dorr- 
Bremme, 1983). Assessment data can then be analyzed in a meaningful and 
ongoing way to help educators define learning goals for students as they 
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plan instruction (Brady 8c McColl, 2010). This type of formative evaluation 
practice is more personal in nature and serves several functions including: as¬ 
sisting in planning lessons, grouping students for differentiating instruction, 
targeting individual student strengths and weaknesses, and (perhaps most 
importantly) adapting instruction to meet students’ academic performance 
needs as the school year progresses (Herman 8c Dorr-Bremme, 1983; Stig- 
gins 8c Bridgeford, 1985). Data informed instruction is also well suited to the 
philosophies and constructs of Catholic schools, as according to the USCCB, 
“One of the strengths of Catholic schools is that there is great latitude at the 
local level related to standards, curriculum, textbooks, teaching methods and 
implementation in the classroom (2014, p.4)”. 

Currently many Catholic schools participate in a type of benchmark 
assessment and evaluation process. Such Catholic schools often use these 
assessments, typically administered in the fall semester, to better identify 
student needs prior to starting the fall educational programming. This data 
assists the Catholic schools and educational practitioners with identifying the 
needs of their student scholars earlier rather than later during the academic 
year. While such practices are helpful and data from such testing is utilized 
to inform instructional planning, this does not meet the standards of a DII 
model. More can and should be done to instruct Catholic school teachers 
and administrators regarding how to expand their assessment programs to 
better conform to the evidence based, DII model. 

The remaining sections will systematically outline how to set up a profes¬ 
sional development to inform, educate, and gain buy-in from teachers sur¬ 
rounding the use of data-informed instruction in Catholic classrooms and 
across a diocese. The steps that will be addressed include the need to change 
perceptions, inform teachers on how to collect and examine data (behavioral 
and educational), and end with ways to help make data collection and analy¬ 
sis an efficient component of daily routines. 

Power of Perceptions 

Traditionally teachers have based their instructional decisions largely on 
“experience, intuition, and anecdotal information (or professional judgment)” 
rather than systematically gathered data (Ingram, Louis, 8c Schroeder, 2004, 
p. 1281 as cited in Young 8c Kim, 2010, p. 13). Teachers’ perceptions have been 
reported as the filter through which new teaching methods are interpreted 
and carried out during instruction (Borko et al., 1997). This may be a distinct 
barrier in moving teachers toward the use of systematic data collection and 
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analysis, as teachers have become accustomed to relying on their own intu¬ 
itions when shaping or preparing instructional lessons (Young 8c Kim, 2010). 

It is perhaps this intuition; however, that might be tapped in an effort 
to obtain teacher buy-in when seeking to change their instructional founda¬ 
tions from intuition to data driven instructional practices. The research of 
Young 8c Kim (2010) has shown that tapping into teachers’ views can be a 
plausible avenue to promote follow-through on the way to data-informed 
continuous improvement efforts. These authors further reported the extent 
to which data effectively advance instructional practice in schools is greatly 
influenced by educators’ pedagogical view, in addition to the relevancy, useful¬ 
ness, and accessibility of the data. 

Additionally, according to studies by Borko et al. (1997) and Mayfield, 
Marion, Flexer, 8c Cumbo (1997), the impact of teachers’ ideas and attitudes 
are strongly linked to their instructional practices. Borko et al. (1997) found 
teachers’ perceptions to be important to the generalization of training effects, 
and therefore stated that explicit attention to teachers’ experiences in addition 
to content practices included in any training, should be a mandatory prior¬ 
ity. It might be hypothesized then, that if training were to enhance teachers’ 
perceptions of a method, they may be more likely to implement the method 
in their classrooms. Professional development and programming for teach¬ 
ers should therefore begin with methods that incorporate teacher percep¬ 
tions into the training process as a way to obtain teacher buy-in. Such buy-in 
may be influenced by the logical application of research already found in the 
literature. Jasper, Hunter and Williamson (2015) noted that data collection 
and analysis is essential if teachers are to avoid returning to strategies that are 
analogous to “...aimlessly firing behavioral and instructional strategies and 
interventions toward the moving targets of student achievement.” (p.??) As 
teachers struggle to apply lesson solutions to complex educational standards, 
they must understand that data is the marker upon which lesson choices can 
most efficiently be made (Jasper, Hunter and Williamson, 2015). 

How to Collect and Examine Data 

Curriculum-based measurement. Curriculum Based Measurement 
(CBM) is one form of DII and has been recognized as one of the most ef¬ 
fective means of formative assessment related to student achievement and 
obtaining an accurate account of student performance across the curriculum 
(Capizzi 8c Fuchs, 2005; Stecker, Fuchs, 8c Fuchs, 2005). CBM can be char¬ 
acterized as any set of measurement procedures that use direct observation 
and systematic, pre-determined, research based guidelines to assess student 
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learning against school curriculum standards (Deno, 1985). Specifically, CBM 
includes evidence-based guidelines that focus on time (duration of the assess¬ 
ment, latency between questions posed and responses received, and inter-trial 
response times); integrity of the implementation of the assessment including 
scripted step by step instructions; normative data for comparison to the indi¬ 
vidual testers’ scores; and pre-made assessment tools (if needed). The design 
of CBM procedures allow for frequent assessment and repeated measures that 
directly tie to possible academic interventions. Many features of CBM have 
been shown to have advantages as a method of data informed instruction. Ac¬ 
cording to McLoughlin and Lewis (2004), these advantages include: 

(a) immediate, accurate, and concrete positive feedback... to teachers, 
students and parents when students are experiencing gains; (b) rapid 
identification of negative performance trends allows a teacher to quick¬ 
ly make responsive changes in students’ programs; (c) and graphed re¬ 
sults can be used to judge whether or not an intervention made in a 
student’s program is having the desired effect, and respond accordingly. 

(p. 167) 

Gansle et al. (2004) stated, “CBM provides educators with a stronger link 
between assessment and instruction than do standardized tests of achieve¬ 
ment” (p. 291), making CBM an ideal choice for a formative assessment that 
can be used to guide students at his or her own pace so that he or she can 
be successful as they matriculate from one grade to the next. Again, this is a 
more personalized form of DII but fits within the philosophy espoused by 
the United States Conference of Catholic Bishops (2014). 

Catholic schools, and the teachers within Catholic schools, typically 
take account of the academic environment in which they find them¬ 
selves both nationally and locally. This allows teachers to prepare and 
challenge students who will be transferring to secondary and higher 
education institutions. (United States Conference of Catholic Bishops, 
2014, p. 5) 

Incorporating Data into the Routine 

Blending data collection and analysis into the daily routine is an inte¬ 
gral part of ensuring that data driven decisions are being utilized within the 
context of the classroom. Assisting teachers with identifying ways in which 
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to blend data collection and data analysis into their routines will help teach¬ 
ers to feel more autonomous in the required assessment processes. Initially 
teachers will be overwhelmed by the concept of adding an additional task to 
their daily activities; however, teaching effective ways in which to embed data 
collection practices into the daily routine will prove to enhance classroom 
programming and decrease the stress level of the teacher. 

In a 2007 study conducted by National Educational Technology Trends 
survey (NETTS) the findings indicated that only 39% of the teachers that 
had access to data systems within their districts and received professional 
development about the data systems felt competent in using the data collec¬ 
tion system. Today, less complex yet more robust systems can be utilized by 
teachers to collect student assessment data. Advances in hand held, real time 
data collection technologies often utilizing tablet-computing appliances are 
only now beginning to be studied. Preliminary results of such studies seem to 
indicate that these devices are having a positive impact on student outcomes 
(Pilgrim, Beldsoe 8e Reily, 2012) and likely influence teaching efficiencies 
(Jasper, Hunter 8c Williamson, 2015). 

Technologies, while efficient and effective at collecting and examining 
data to make curricular and methodology decisions, can be expensive to cash 
strapped Catholic schools. For this reason, utilizing free or near free services 
available from entities such as Google for Education (https://www.google. 
com/edu/) or excel or chartdog, which is accessible through www.interven- 
tioncentral.com may be cost effective options. 

This Google service for example, allows for the use of low cost comput¬ 
ing hardware such as the ChromeBook (starting at $25o.USD as of this 
writing) to be used in coordination with cloud based, collaborative, and free 
applications such as: Google Forms, Google Docs, and Google Sheets. These 
applications can be collaboratively applied to create easy to complete data 
recording documents that collect, organize and chart data. This data can then 
be easily entered from component parts (see Figure 1) and aggregated into 
information regarding the overall achievement trajectory of an entire school 
or diocese (see Figure 2). 
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Sample CBM form 

T Required 


Words per minute? * 




MuLT.tr „trjtu7nV ji.’jviLwtrrfr,' llimurjh Grxigki finnn.i 


PfhA'fcrn-rt tty THtnuntorf KrnttiwawMinur«Hha-iiad liy Guuyhr 

5 Googkrcritli HffmttttMiM ■ tHfliH nrSMvw*. AMAnrallwra 


Figure 1. Individual student form (sample) 



Figure 2. Class aggregate CBM averages data graph with intervention line (Sample) 

Figure 3 is an example of a graph generated through excel that illustrates 
an individual student’s performance. In the figures, you will notice, the verti¬ 
cal axis represents the skill measured and the horizontal depicts the time. 
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Baseline Intervention 


Math Intervention 



Days 


■Goal 


Aim 


■ Number of Problems Correct 


- Linear (Number of Problems 
Correct) 


Figure 3. Example math intervention graph 


As shown in the figures, data collection and monitoring does not need to 
be complex. Each example shown involves very little training and often such 
graphical displays can be automatically generated from pre-existing templates 
(https://drive.google.com/templates?q=gradebook&sort). Still, any data 
collection and analysis methodology will need to be trained, reviewed, mod¬ 
eled, and monitored with continuous corrective feedback given in order to 
ensure that data collection practices are being implemented with fidelity. In 
other words, interventions and individualized instruction are only effective if 
implemented as intended. The more support that teachers have in embedding 
these data collection practices into their classrooms, the more likely they are 
to attempt these practices on a daily basis. 

Data can be collected on academic and behavioral needs of the student, 
and it can be collected on individual students, the classroom as a whole, or 
across classrooms. Figure 4 depicts classroom averages. 
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Math Class Averages 



Months 


■Class 1 
■ f.lass ? 
■Class 3 


Figure 4. Example math classroom average graph 

It is also important to note that schools that utilize data systems that are 
integrated with existing systems of data collection, for both behavioral and 
academic data, will decrease the stress that teachers have with embedding 
these practices into their classrooms. The 2007 NETTS study also identified 
a barrier to data collection management and use by teaching clinicians was 
largely evident due to the lack of integrated systems within the school dis¬ 
trict. 

Not only is it important to collect data on academic performance but it 
is also essential to collect data on behavior. Comparing periods of off task 
behaviors to on-task behaviors during academic work times can shed ad¬ 
ditional information onto the grades, particularly if you notice fluctuations. 
For example if a child is on-task and working hard throughout the assign¬ 
ment but his or her grades are falling this may indicate a “Can’t do” problem. 
If grades are falling and data reveal that the behaviors are primarily off task, 
then this may suggest a “Won’t do” issue. Both of these, “Can’t do” and “won’t 
do” are significant barriers to success but both require a different approach to 
remediation. 

Ensuring that teachers are trained efficiently in understanding the differ¬ 
ent type of data collection procedures and practices is an important part of 
professional development. Teachers should be informed and trained on the 
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different types of data collection methods, when each data collection would 
be appropriate to use, and provided step-by-step coaching and instruction on 
these practices (refer to the paragraphs below on frequency, duration, latency, 
and time sampling). This should be done whether using traditional paper and 
pencil techniques or when using more robust electronic means of taking and 
keeping data. Once these steps are completed, fidelity checks on data collec¬ 
tion practices should also be conducted to monitor teacher progress towards 
data collection methods. Often times, the best method to monitor teacher 
fidelity in data collection practices is to collect the Inter-observer agreement 
(IOA) data of a particular data collection sample. This is a method in which 
the teacher and a third party individual take data during the same time 
period, analyze the same behavior, use the same data collection method, and 
an calculate an analysis of agreement between observers for that data. This is 
helpful in identifying if additional coaching and professional development is 
needed to remediate any issues within the data collection practice procedures. 
Data is not helpful if it is skewed or inaccurate. Below are a few tips on 
teaching specific data collection practices. 

Momentary time sampling. Momentary time sampling is also known 
as an interval recording method. This data collection method requires the 
observer to determine whether a behavior occurs or does not occur during 
specific time frame. Once the length of an observation session is identified, 
the time is broken down into smaller intervals that are all equal in length. For 
example, a 30-minute observational session may be separated into 30 inter¬ 
vals that are one minute in length. The observer simply makes a mark in the 
identified interval if the targeted behavior occurs or doesn’t occur depend¬ 
ing on the data being collected, at the very end of the identified interval. A 
number of different tools can be used to assist the observer with the begin¬ 
ning and end of a designated interval. Some examples are a kitchen timer, 
an alarm on a hand held watch, or a tape recording with a sound. The utility 
of this method is that is not labor intensive and allows one to collect data on 
multiple behaviors or skills. 

Frequency data collection. Frequency data collection is simply having 
the observer count how many times a behavior occurs during a designated 
period of time. Those designated periods might be a minute, an hour, a day, 
or a week. This data collection is most useful with behaviors that are discrete 
(definitive beginning and end) and short in duration (e.g., number of curse 
words, number of short talk-outs without raising hand), or are permanent 
products that the student has created (e.g., number of correct math problems, 
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number of homework assignments submitted). Collecting frequency data can 
be time consuming, as it requires a great deal of attention; however, if done 
correctly, it yields data that is valuable and exact. 

Duration data collection. This data collection method monitors the 
amount of time that a behavior occurs during the observation period. Specifi¬ 
cally it is used for behaviors that last for more than a few seconds and/or for 
varying lengths of time (e.g., on-task or a tantrum in the classroom). This data 
collection method is best completed with the use of a timer, stop watch, etc. 
Collecting the duration of behavior is important when the goal is to shorten 
or lengthen the behavior. For example, some behaviors such as on-task are be¬ 
haviors that are desired but not occurring to the extent needed and a baseline 
measure of initial time allows one to see gains more quickly. 

Latency data collection. Latency refers to the time between a prompt and 
compliance. Often students display the correct behavior but the length be¬ 
tween demonstrating the behavior and the request is delayed. In instances 
when compliance is slow, collecting latency data can assist with shaping the 
behavior to shorter, more appropriate response times. 

Collecting and Using Class-wide Academic data 

Using formative data. Ongoing data collection, whether academic or be¬ 
havioral, represents the cornerstone of all successful classrooms. With regard 
to academics, based on the student’s formative assessment data, the teacher can 
make determinations about restructuring assignments to meet the needs of 
the student. As seen in figure 2, this teacher changed her methodology as in¬ 
dicated by the dotted intervention line. After changing her methodology, she 
then saw an increase in the rate at which her students achieved. Only through 
the visual analysis of the early data as aggregated together using Google Forms 
and Sheets, could the teacher note the slow progress and change her methodol¬ 
ogy accordingly. While figure 1 and 2 portray a fictional example of the visual 
power of graphed student formative assessment data, the idea can go consider¬ 
ably deeper. Using the Google apps, this teacher could drill down to individual 
student data, find trends within groups, formulate groupings for differentiated 
instruction using group analysis trends or even serve up her students’ data to be 
analyzed similarly by school administrators or diocese officials. The trick is to 
make the data collection as simple as possible so as to not burden the teacher 
while also making the data easy to organize and visualize for daily use. Google 
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apps for education is only one low cost example for doing this. Numerous soft¬ 
ware applications are available that essentially do this same thing making the 
process of implementing DII relatively easy compared to years past. 

Conclusion 

If Catholic schools are to remain competitive with other public and pri¬ 
vate educational institutions, it is important that they can track and dem¬ 
onstrate the academic achievement of their students using data. Thankfully, 
showing this achievement is a relatively simple process when DII is used as a 
primary methodology for evaluating and modifying instruction. It is impor¬ 
tant for both leaders and teachers within all educational systems to under¬ 
stand the pedagogical practices that will enhance educational outcomes for 
their students and DII is fundamental to this understanding. Leaders must 
utilize these practices and disseminate information in a way that will encour¬ 
age teachers to use evidence-based assessment, analysis, and interventions in 
the applied practice setting. 

Much of this has already been taking place within the public school and 
charter school realm through collaborative efforts and on-going professional 
development opportunities. This is due, in large part, to legal mandates and 
an increased focus on accountability. Regardless of the reason, professional 
developments with this focus have proven fruitful and teachers as well as 
students are benefiting from the new trend of using DII. As Catholic schools 
are in competition with these entities, similar actions should be taken to 
ensure that they remain relevant and viable options to families seeking a 
Catholic focused education that also maintains a high level of academic ac¬ 
countability and rigor. 
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